There is an emerging market in the information technology sector that focuses on the reading of brain waves in order to operate devices or to generate speech communication. Devices that read brain waves and control devices are commonly referred to as brain-computer interfaces (BCI), mind-machine interfaces (MMI), direct neural interfaces (DNI), synthetic telepathy interfaces (STI) or a brain-machine interfaces (BMI).
A BCI is a system in which messages or commands that a user sends to the external world do not pass through the brain's normal output pathways of peripheral nerves and muscles. For example, in an electroencephalography (EEG)-based BCI, the messages are encoded in EEG activity. A BCI provides its user with an alternative method for acting on the world. For more information on BCIs, see Wolpaw, Jonathan R., N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain-computer interfaces for communication and control,” Clinical Neurophysiology, 113 (2002) 767-791, which is hereby incorporated by reference in its entirety.
A BCI changes electrophysiological signals from mere reflections of central nervous system (CNS) activity into the intended products of that activity: messages and commands that act on the world. It changes a signal such as an EEG rhythm or a neuronal firing rate from a reflection of brain function into the end product of that function: an output that, like output in conventional neuromuscular channels, accomplishes the user's intent. Id. As such, a BCI replaces nerves and muscles and the movements they produce with electrophysiological signals and the hardware and software that translate those signals into actions. Id. A BCI operation depends on the interaction of two adaptive controllers: the user's brain, which produces the signals measured by the BCI; and the BCI itself, which translates these signals into specific commands Id.
Currently, devices such as EEG headsets have become readily available to the retail market. These devices read the brain's six wave types, or “bands”, and classifies them into one of the following categories:
TABLE 1The six (6) wave types emitted by a typical brainduring normal thought processes, including theirfrequency ranges and normal correlative behavior.BANDFREQUENCY (Hz)NORMALDeltaup to 4Adult slow-wave sleep;In babies;Present during “continuousattention” tasks;Theta4-7 Drowsiness;Arousal in adults;IdlingAlpha8-12Relaxation/Reflection;Closing the eyes;Inhibition controlBeta13-30 Alertness;Active or anxious thinkingGamma 30-100+Multiple perceptions, suchas “sight and sound”;Short-term memorymatching of recognizedobjects;Mu8-13Rest-state motor neurons
EEG devices read the bands of brain activity which present themselves as electrical waves. Typically, these waves are read in real-time, interpreted and then associated to a pre-defined action resulting in a “one-to-one correlation”, meaning the user thinks “X” and the device either speaks “X” or executes “X”.
Devices exist that can convert a user's thought to a direction (e.g. a direction along the axis) or movement (e.g., movement of a body part of a user, or movement of an external object), in that a user imagines a movement or direction, such as “left”, or “ball moves up”, or “hand moves down”, which creates an easily identifiable wave pattern for the EEG to read, and the EEG sends this pattern to proprietary software which can then send a suitable corresponding (e.g. “left”) command to something as simple as a computer cursor or mouse pointer to something physical, such as a wheelchair, car or other device, or can use the thought as a component into an output of item of information.
Examples include that a user thinks “left” which results in something moving left (e.g., a prosthetic appendage, a mouse pointer, etc.).
There are inherent challenges in the existing modality of one-to-one correlations with regards to the association of brain wave patterns in spoken language or executed actions. The only way researchers have been successful with accurate and fast thought-to-speech technologies has been to implant computer chips within the brains of the users, often involving open-brain surgery. The existing thought-to-speech technologies often attempt to identify common/frequent “patterns” within the brain waves of spoken and “non-spoken/thought/imagined” language. Many times, these patterns can be the same for words that are similar cat, rat, bat, mat, etc. Or, similar phrases such as, “I hate cats” and “irate bats”. Even with implanted chips, users can expect a success rate of between 45% and 89%. For more on thought-to-speech, see “Machine Translates Thoughts into Speech in Real Time”, Dec. 21, 2009, “phys.org”, which is hereby incorporated by reference in its entirety.
If an “over-the-counter” EEG machine is incorporated with the existing modalities of brain wave interpretation, the resolution of the device is typically not accurate enough to identify the intended thought consistently to be useful in day-to-day practice. Not only is the general interpretation of brain waves and specific brain wave patterns very difficult due to their dynamic nature and high variability, but outside electrical interference from commonly used devices, such as cell phones and televisions, make this process even more difficult and the output more inaccurate.
The challenges in the user's thought-to-command category of thought-controlled devices are nearly identical with the above thought-to-speech category. Problems with correctly identifying the non-verbalized command greatly affect the selection of the associated action.
There are also challenges with the user's thought-to-direction category. These one-to-one correlations of non-spoken commands/thoughts into the direction of “something” is by far the most accurate and repeatable process for all thought-controlled devices. Up, down, left and right are all consistent, repeatable patterns, and might be the most easily identifiable markers within commonly collected EEG brain wave data. However, this category is limited in that researchers and inventors are only using this logic to associate directional-based thoughts with the movement of an object (e.g., movement of a mouse pointer, movement of a computer cursor, operation of a wheelchair, driving a car, operating an exoskeleton, or the like), which remains a one-to-one correlation of “imagined direction” with the actual executed direction, whether it be virtual via software or mechanical via devices and equipment.
This challenge/limitation leaves a huge void in controlling devices that are not operated with directional thought. For example, controlling, using thought, devices such as televisions, doors, appliances, adjustable beds or even mobile phones and tablet computers is not easily done using conventional methods. The answer would lead one to operate within the thought-to-speech or thought-to-command modalities, which are highly unreliable and unpredictable. It would not necessarily lead one to operate within the thought-to-direction modality due to the traditionally perceived limitation of a finite number of directional thoughts.
As can be seen, there is a need for improved methods for using thoughts and other input to define a single action, function or execution for non-tactile (e.g., not requiring touch input as is required with a keyboard or mouse) or thought-controlled devices that is more reliable and accurate.